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Abstract 

We discuss level crossing of the free-energy of TAP solutions under variations of external parameters 
such as magnetic field or temperature in mean-field spin-glass models that exhibit one-step Replica- 
Symmetry-Breaking (1RSB). We study the problem through a generalized complexity that describes 
the density of TAP solutions at a given value of the free-energy and a given value of the extensive 
quantity conjugate to the external parameter. We show that variations of the external parameter 
by any finite amount can induce level crossing between groups of TAP states whose free-energies are 
extensively different. In models with 1RSB, this means strong chaos with respect to the perturbation. 
The linear-response induced by extensive level crossing is self-averaging and its value matches precisely 
with the disorder-average of the non self-averaging anomaly computed from the 2nd moment of thermal 
fluctuations between low-lying, almost degenerate TAP states. We present an analytical recipe to 
compute the generalized complexity and test the scenario on the spherical multi-p spin models under 
variation of temperature. 

1 Introduction 

Mean-Field Spin-Glass Models like the Sherrington-Kirkpatrick model or the p-spin models are known to 
have a very complicated phase space with many metastable states. An important physical consequence is 
that the system fluctuates not only within each equilibrium state but also among different equilibrium states 
whose free-energies are sufficiently low and close to each other. The non-trivial nature of the fluctuations 
among the low-lying states have been fully uncovered by the powerful theoretical tools, i.e. the TAP, replica 
and cavity methods p^. The presence of many states leads also to the so called chaos problem, i.e. the 
question of whether equilibrium states at different values of the external parameters such as magnetic field or 
temperature are correlated or not [2] • In the present paper we discuss the problem from a TAP perspective. 
The states are usually identified with solutions of the TAP equations 1 ; if a given TAP solution has a 
non-vanishing Hessian it can be continued analytically upon a change of the external parameter. We will 
say that two states at different external parameter coincides if one is the analytical continuation of the 
other . A question related to chaos is to know whether the equilibrium TAP states at a given values of, say, 
magnetic field h are the same (in the sense of the analytical continuation) of those at a different value of 
h. If this is the case chaos is certainly not present. Instead if the states are not the same we derive chaos 
provided we assume that different states are not correlated; this is surely the case in 1RSB models (because 
by definition different states are minimally correlated) but can be more complicated for FRSB models. In 
1RSB systems the TAP states with free energies below the threshed values have a non-vanishing Hessian 
therefore each of them can be analytically continued upon changing the external parameters. Therefore the 
equilibrium states at the new value of the the external parameters must have been already present as some 
TAP states at the old values and they can be identified considering the evolution of the old TAP states 
by the variation of the parameters. We show that to describe the evolution of the TAP states we must 
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consider a generalized complexity which represents the density of TAP states at a given value of free-energy 
(per spin) / and of an extensive quantity (per spin) y = Y/N, where N is the number of spins, conjugate 
to the external parameter h y to be varied, i.e. magnetization for the magnetic field and entropy for the 
temperature. 

In general at fixed values of the external parameters the typical states with a given value of / have 
a definite value of y but there are also TAP states with the same / and different values of y although 
with lower complexity than the typical ones. Thus in general the function £(/, y) is non-trivial. Assuming 
the existence of this function we draw the following conclusions. We prove that variation of the external 
parameters by any finite amount induces level crossing of the free-energies of TAP states at extensive levels. 
Thus the equilibrium TAP states at different values of the external parameter are different. Furthermore 
from the function £(/, y) we can compute the induced inter-state linear-response; it turns out to be self- 
averaging and its value matches precisely with the value predicted by the analysis of spontaneous thermal 
fluctuations through the FDT. We present an analytical recipe to compute the generalized complexity and 
present explicit calculation for some specific cases. In particular we show the existence of the function 
£(/, m) for generic FRSB and 1RSB models. We also consider the entropy-free-energy function £(/, s) 
(related to the behavior under temperature changes) in 1RSB spherical p-spin models. This function exists 
for 1RSB spherical model with multiple p-spin interaction implying chaos in temperature while its support 
shrinks to a single line in the (/, s) plane in the limit of a single p-spin interaction consistently with the 
absence of chaos in temperature in this case. 

The problem of level-crossing of TAP states has been recognized in earlier works, for instance in Ref. 
0] 03 and others. In particular level crossing of individual TAP states upon infinitesimal changes in the 
values of the magnetic field (5h = 0(1 /s/N)) was observed. In the present paper we are interested instead 
with evolution of TAP states under small but finite changes in the external parameters, i.e. changes that 
induce extensive variation of the free energy. We want to know if the set of equilibrium states at a given 
value of the external parameters contains as a whole the same set of equilibriums states at different values; 
note that this does not exclude the possibility of some internal reshuffling of the relative weights of the 
states. If only the latter happens we would have just some mild, sub-extensive level crossing between the 
states but no chaos. 

More recently Krzakala and Martin (KM) [5] studied the level crossing phenomena in an extended 
version of the random energy model|xj in which each state has a random energy and a random extensive 
variable conjugate to an external parameter, such as temperature. Both random variables are assumed 
to follow Gaussian distributions. Based on the phenomenological model they provided a very interesting 
general phenomenology on the chaos problem. The generalized complexity we study in the present paper 
provides a firmer ground for their picture. 

The plan of the paper is the following. In section 2. we review previous works related to the present 
paper. In section 3. we introduce the generalized complexity. We discuss its evolution under variation of 
external parameters and explain its physical consequences. In section 4 we present an explicit calculation 
of the evolution of the generalized complexity of a spherical multi-p spin model under variation of the 
temperature. At the end we discuss our results. 



2 Intra-state and Inter-state Susceptibility 

A well known effect of RSB is the difference between the susceptibility inside a state and the true thermo- 
dynamical susceptibility For example, the magnetic susceptibility inside a state a in zero magnetic field is 
given by 

Xa=P(l-qBA) (1) 

where qea is the Edwards- Anderson order parameter, while the actual magnetic susceptibility of the system 
is given, according to the Parisi solution yQ, by: 

X = P(l-q) (2) 

where q is the average of the overlap between replicas. De Dominicis and Young [H] have shown that this 
is a consequence of the presence of many states, so that in the application of the fluctuation-dissipation- 
thcorem there is a new term which takes into account the fluctuations of the magnetizations over different 
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states. They assumed that the free energy of the system is given by a sum over all TAP solutions weighted 
with their free energy: 

a 

then the susceptibility to a change in a given external field h y (e.g. temperature or magnetic field) reads: 

d 2 1 



x 



dhl (3N ^ 



where the square brackets mean Boltzmann average over the states 

and y a is the value on state a of the parameter conjugated to h y , (e.g. magnetization or entropy per spin): 

Va= dh y (6) 

The first term is the susceptibility of a state while the second term is the fluctuation over the states of the 
parameter y a . The first term gives a contribution of (3(1 — Qea) while the second term can be written as 
(3(qEA —q) so that the correct result J5J for the susceptibility is recovered. Another interesting feature of 
the susceptibility is that the susceptibility on a given sample, defined in terms of correlation functions of 
the spontaneous thermal fluctuations is not self-averaging. This has been pointed out by Young, Bray and 
Moore in Ref. p5 where they studied the magnetic susceptibility, 

XJ = ^T l (.(SiS j )-(Si)(8 j )). (7) 

y 

Here s, (i = 1, . . . , N) are the spin variables. The non-self averageness was interpreted as an effect of the 
presence of many states, with sample-dependent 0(1) free-energy differences between those that dominate 
the equilibrium measure at low temperatures. This interpretation is confirmed noting that the TAP suscep- 
tibility defined above eq. 10} , (which is defined differently from Xj) is indeed not self- averaging, as we show 
in appendix B. In particular its disorder variance is the same as that of \J computed in The problem 
is that the total magnetization and susceptibility, derived from thermodynamic derivatives of free-energy 
which itself is self-averaging, should be self-averaging. Thus one faces with an apparent contradiction. Then 
what is the true response? Some earlier numerical studies on finite size SK model by exact enumeration 
method provide a useful insight on this problem. As shown in Ref. ,4 and the magnetization per spin 
mj(h) of a given sample grows in a step- wise manner under increasing magnetic field h at low temperatures 
(See Fig. 2. of ,5 4 ). The spacing between each steps and height of each step, varies from step to step and 
sample to sample [|J. Note that this is consistent with the fact that fluctuations are not self-averaging since 
the linear-susceptibility defined as x.J = lim^^o Smj(h)/5h is related by FDT to the fluctuations. In Ref. 
5 Young, Bray and Moore suggested that the step-wise response is due to level-crossing of TAP states. 
Furthermore, they conjectured that the typical separation between the steps is of order 0(1/ yN) and that 
the profile converges to a unique limiting curve in the thermodynamic limit m(h) = limjv^oo rnj(h). So 
m(h) is self-averaging and thus the linear-susceptibility defined as x — A m A/i^o Am(h)/Ah is also self- 
averaging. It is reasonable to expect that x matches with disorder average of xj- Note that Sh and Ah 
used in the definitions of xj and x are a f completely different scales. While Sh must be chosen smaller 
than the typical spacing between the steps, which is likely to be of order 0(1/ V~N), Ah can be chosen to 
be arbitrary small but fixed when the thermodynamic limit N — > oo is taken. 

According to the following argument by G. Parisi 0, the difference between the susceptibility and the 
intra-state susceptibility in general implies that the equilibrium states at different values of the external 
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parameter cannot be the same. Indeed from eq. J2J), it follows that the magnetization of the equilibrium 
states in presence of a small but finite magnetic field h becomes, 

m~j0(l-g)ft. (8) 

On the other hand the analytical continuation of the old equilibrium states would develop a smaller mag- 
netization /3(1 — qEA)h where qea is the Edwards- Anderson order parameter. Therefore the equilibrium 
states in presence of a small but finite field h had a non-zero magnetization per spin even in the absence of 
the field 

m ~ (3(q E A - q)h. (9) 

Therefore the new equilibrium states cannot be the analytical continuation of the old equilibrium states. 

Here an important point is that h is chosen arbitrary small but fixed when the thermodynamic limit 
./V — > oo is taken, i.e. h is at the scale of Ah and not of Sh. In particular at this scale we have no problems 
of lack of self-averageness. As we explained in the introduction, we are interested in extensive level crossing 
therefore in the following we are going to consider always variations in the external parameter at scale Ah. 
Note that this argument can be applied whenever the susceptibility to a given field h y is different from the 
intrastate susceptibility, i.e. whenever the fluctuation of the conjugated parameter y, i.e. the second term 
in eq. (@J), is not zero. The fluctuations obviously vanish if there is only one state. 



3 Extensive Level Crossings 

The presence of metastable states with extensive non-zero magnetization in zero field may appear rather 
counter-intuitive, however in the TAP context their number can be computed and one can show that there 
is an exponential number of solutions with non zero magnetization, although with a smaller complexity 
with respect to the solutions with zero magnetization. In order to have a deeper look into the evolution of 
the phase space we consider a generalized complexity i.e. the logarithm of the number of TAP solutions 
with given values of the free energy and of the magnetization: 

£(/,m) = lln^(m a -m)<5(/-/ a ) . (10) 

a 

We want to study the evolution of the curve £(/, m) under the application of a magnetic field. 




Figure 1: Complexity of the TAP solution as a function of the free energy and magnetization per spin 



In figure ^ we show a schematic plot of the function £(/, m) near the lower-band edge where the 
equilibrium states are located. Near the equilibrium states and below the critical temperature, the function 
can be expanded as: 

E(/,m)=/?i/-am 2 (11) 

where x is the Parisi parameter and a is some parameter to be determined, and we have shifted the free 
energy so that the equilibrium free energy in zero field is zero. Let us emphasize that the function £(/, m) 
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is, by definition, an extensive self-averaging quantity. Now we want to consider the evolution of the states 
on this curve when the small field h is switch on. The free energy of a state will be modified according to: 



by definition we have m a — ~df a /dh therefore the new free energy of the set of TAP solutions with given 
values of / and m is given by: 

f = f-mh+l^h 2 +0{h 3 ) (13) 



their magnetization is given by: 



2 dh 2 

™' = m + + 0(h 2 ) (14) 

Expressing through eq. (|11H the free energy in terms of the complexity c and the magnetic field we get: 

f = f x+ bm* (15) 

where we defined 

b = a/ fix. (16) 

Putting this expression into eq. (|13fl . we obtain the new value (after the field is switched on) of the free 
energy of the states that in zero field had complexity c and magnetization m: 



1 d 2 f a 

2 dh 2 ' ' ' (3x 



f' = bm 2 -mh+-—^h 2 +0(h 3 ) + — (17) 



The new equilibrium states are those that minimize /'. First of all we note that the minimum with respect 
to c is obtained for c = 0, this is consistent with the fact that the equilibrium states under any circumstance 
below the critical temperature should always have zero complexity, and therefore the zero-field TAP states 
that arc candidate to become equilibrium states in a field must must have zero complexity. Thus we are 
interested in the evolution of the equilibrium states along the zero-complexity line / = bm 2 . In order to 
minimize /' with respect to m we note that the third term in eq. Ijl7|l in principle depends on m, but for 
values of m of order O(h) this variation is basically a third order effect, therefore at second order in h it 
can be considered as a constant. Then we obtain: 

^- = 26m -h = 0^m= — (18) 
dm 2b v ; 

Thus the equilibrium states in presence of a field are the states that had a non-zero magnetization m = h/2b 
in zero field and the evolution of the TAP states is driven by extensive level crossing, indeed the free energy 
difference between these states was A/ = h 2 /4b in zero field while it becomes negative A/ = —h 2 /4b 
in presence of a field. This is the same result obtained above: in presence of a field the TAP solutions 
with lowest free energy are not the continuation of the TAP solutions with lowest free energy in zero-field. 
Accordingly the magnetization is given by: 



and the full linear-susceptibility is given by: 



In 1RSB models TAP states with extensive difference in the free-energy must have zero overlap with 
respect to each other, i.e. they are totally uncorrelated. Thus extensive level-crossing automatically means 
strong chaos in 1RSB systems. 

The basic assumption of this derivation is the existence of the zero-complexity curve / = 6 m 2 , which 
follows from the existence of the function £(/, m). Once the existence of this function is assumed the 
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non-trivial result is that the evolution of the TAP states under a change in the magnetic field is driven by 
extensive level crossing. As such the previous derivation can be extended to any couple (h y , y) representing 
an external field and its conjugated extensive variable, e.g. temperature and entropy provided the zero- 
complexity curve / = by 2 exists. This assumption is equivalent to the assumption of the previous section 
that the parameter y a fluctuates over the states. The connection with the result of the previous section 
can be established also at a quantitative level by showing that the two expressions for the susceptibility eq. 
(I20[l and eq. are equivalent. In order to do that we introduce the function: 



$( Ay ) = 1 ln£ e -PNf a + X v Ny a (21) 



This is a summation over all TAP states with a weight which depends also on the value of Y = Ny, when 
X y = it reduces to the Boltzmann weight such that $(0) is minus the free energy. ^From the definition 
follows that 



d 2 $ 
dXj 



m(yl) - (i/aH (22) 



A„=0 



On the other hand using the generalized complexity through eq. (|15|l we can write 

$(A) = max c , y (c - (3{by 2 + + X y y). (23) 
Again the maximum is at c = and the maximization with respect to y gives 

§ - = w (24) 

which is linear with respect to X y . Using eqs. lj2~^ and lt22l we get: 



1 ax 



This equation together with eq. 122f) prove the equivalence between eqs. (|20|) and (@J for the susceptibility 
that can be written as: 

a d 2 <S> 

Xy = Xya + P 



dx 2 v 



(26) 

A„=0 



where the first term is the generalized susceptibility inside a state, e.g. the specific heat if h y is the 
temperature and y is the entropy. Notice that we do not need to compute the intra-state susceptibility to 
infer the picture, it is sufficient to check the existence of the zero-complexity line. 

In appendix A we report the general method to compute the function <I>(A) for a generic model. In 
particular in the case of the magnetic field we can show that the second derivative of $(A) has the correct 
value needed to recover the right TAP susceptibility in either FRSB and 1RSB models: 



<9 2 $ 



dXl 



= qea - q (27) 

A m =0 



Note that the derivation of this section assumes that the zero complexity curve f — b m 2 is a self-averaging 
smooth function. Of course at any finite N this curve is actually made of points therefore on sufficiently 
small m scale (i.e. scales that go to zero with some proper power of 1/N) we expect it to have rapid sample- 
to-sample fluctuations around its sample-independent average. These fluctuations and the corresponding 
lack of self-averaging in the r.h.s. of eq. ll'L'l) are irrelevant at the much larger scales which we consider and 
to which the derivation of the present section applies. 



4 Spherical p-spin Models 

In this section we show that picture of the previous section applies to 1RSB spherical p-spin models. 
In particular the presence of chaos in temperature can be univoqucly associated to the behavior of the 
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zero-complexity line as a function of the free energy and of the entropy. Following |13| we consider the 
Hamiltonian: 

N N 

H = - ^2 J h-i P <Tii - e ^2 K h-l T 07i •••0"/ r , (28) 

ii<...<i p ii<...<i r 

where the spins Oi are subject to the spherical constraint er? = A, and the Gaussian random couplings 
Ji 1 ...i p and ii'; 1 ...i r have variance p!/2A rp ~ 1 and r\/2N r ^ 1 . The p + r spherical models may display a 
nontrivial thermodynamic behavior when p > 3 and r — 2: in that case there is a transition between a 
1RSB thermodynamic phase (low e), to a FRSB phase (large e) [TJ. On the contrary, if both p and r are 
strictly larger than two, the model is expected to have a normal 1RSB thermodynamic behavior. This is 
the case we will analyze. In particular we have studied numerically the case p = 3 and r = 4. The TAP 
free energy density is |T5]. 

PJtap= - Ji 1 ...i p m il ---m ip - ejr K h ...i r m h . . .m, lr - -log(l-g) 

- ^[(p-l^-M^ + l] - e^^r-l^-rg^ + l] , (29) 

where rrii = (<Ji) are the local magnetizations, and q is the self-overlap of a state, q — J2% wf/iV. In the case 
of the single p-spin interaction [151 1161 IT7| it is straightforward to see that there is no chaos in temperature. 
Indeed by writing rrii = q x ^ 2 Si where §i is the vector of the angular variables normalized to one, we see 
that the TAP equations for the angular variables do not depend on the temperature, therefore the ordering 
of the states does not change in temperature. The decomposition of the free energy in angular part and 
overlap part breaks down if the model have more than a single p-spin interaction and this could lead to 
chaos in temperature. In particular in |18j the dynamical evolution under temperature changes of the 
TAP states was considered between the dynamical and the critical temperature. We note that with some 
modification the present picture of extensive level crossing can extended also in this region of temperatures 
where the complexity of the equilibrium states is finite. On the other hand chaos in temperature in the 
p + r model below T c can be proven considering the free energy shift between two real replicas forced to 
have a given value of the overlap |19j . Here we want to show that this result can be recovered through the 
study of the entropic zero-complexity line. 

The computation of the complexity of the model l|29|l can be done through standard methods like those 
sketched in the previous section and was presented (up to order e 2 ) in The complexity at fixed value 
of the free energy can be obtained extremising the following effective action with respect to the parameter 
B,T,q and u: 



S = (3u[g(q) + e 2 h(q)-f]+(B 2 -T 2 



ip(p~l)/3V- 2 +e 2 ^-l)/3V 



- \ log Q/?W" 2 + ~e 2 /3W- 2 ) - logT + \{5 2 u 2 (<f + ,V) - ~ 

+ i/3 2 S 2 (pqP- 2 + e 2 rq r - 2 ) + (3{B + T) [A(q) + e 2 C(q)] + ^(3 2 uB (pq^ 1 + eVef" 1 ) , (30) 
and where we used the following definitions, 

g(q) = -^io g (i- q )-^[{p-i)qP-p q "- 1 + i] (3i) 

h(q) = -|[( r -i)g'-_ rg r-i + 1 ] (32) 



dg_ 

drrii 
dh 
drrii 



A(q) m, (33) 
C(q) m, . (34) 
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In order to compute the complexity at given value of the free energy / and of the entropy s we must add 
to (|30|l a term X s s — \ s s(q,f3) and extremise with respect to X s . The function s(q,j3) is the complexity of 
a given solution which can be obtained from eq. 1291 

s(q,P)=-^^ = llog(l- q )-^[(p-l) q P-pq^ + l] e ^[(r-iy_ r(f -i + l] (35) 

The corresponding saddle point equations can be solved numerically. As noted in |13| there are two solutions 
of the saddle point equations, one that is BRST (Becchi-Rouet-Stora-Tyutin) symmetric and another that 
is not. The lower band edge is described by the BRST solution. Numerically we start from this solution and 
consider the complexity of states with entropy different from the equilibrium one. Solving the SP equations 
with respect to B,T,q,u, X s with extra constraint that the complexity is zero yields the zero-complexity 
curve. 

In figure|21we plot the entropic zero-complexity line for a 3 + 4 model at temperature T = .35 < T c and 
at values e = .1 and e = .2. Numerically the second derivative of /(s) in s — s eq diverges as 1/e 2 for e — > 0. 
In this limit, the angular variables can be factorized and the entropy of the states is univoquely determined 
by their free energy; correspondingly the two branches of the zero complexity curve join on a single line 
s = s typ (f), that is the typical complexity of the states with free energy /. Note that since the divergence 
is proportional to e 2 it is consistent to consider the action l|3U|l which is valid at 0(e 2 ). ^From the existence 
of the zero-complexity curve follows that the dominant TAP states at different temperatures are different. 
This implies chaos in temperature because in a 1RSB system different states have vanishing mutual overlap. 
In this context the disappearance of chaos in the limit e — > is determined by the divergence of the second 
derivative of the zero-complexity line. 

F-F«, 

\ 1.4-lfT 7 ' 



\ _„ / 




-0.00001 -5-10" 6 5-10~ 6 0.00001 q 

Figure 2: Zero-complexity line of the free energy as a function of the entropy for the 3 + 4 spherical model 
with e — .1 (dashed line) and e = .2 (continuous line), the second derivative of / (s) in s — s eq diverges as 
1/e 2 for e — > 0, in this limit the model has a single p-spin interaction and chaos in temperature disappears 



5 Discussion 

Our approach applies to all situation in which TAP states at a given value of some external parameter h y 
(e.g. temperature or magnetic field) can be continued analytically at different values of h y . If this is the 
case the knowledge of the states at given value of h y is sufficient to determine the equilibrium values of 
extensive variable y = Y/N in a definite range of values. We have shown that this can be done studying the 
generalized complexity E(/, y). In particular our approach applies to 1RSB models because the Hessian 
of the equilibrium TAP states is non-vanishing. In 1RSB models we could also establish a connection 
between level-crossings and chaos. Thus our results provides firmer grounds for the phenomenological 
picture proposed by Krzakala and Martin in In this context absence of chaos with respect to the 
external parameter h y (magnetic field or temperature) appears when the support of the function £(/, y) 
(y is the parameter conjugated to h y ) shrinks to a single line in the (/, y) plane. 

The application of our approach to FRSB is complicated by the fact that the equilibrium TAP states 
are marginal so in principle we cannot be sure that they can be continued. However one could study the 
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zero-complexity line £(/, y) = 0. In the case of magnetic field this curve certainly exists and have the 
correct slope qEA — q- It would be interesting to check the existence of the zero-complexity line for the 
entropy. This is a further motivation to obtain the quenched solution for the complexity in FRSB model. 
Provided the zero-complexity lines exists in FRSB model for various perturbations it would be interesting 
to know the stability of the corresponding states, we suspect that they are not-marginal. However in FRSB 
the connection with chaos is less clear. In this respect, it would be interesting to check the existence or not 
of the entropic zero-complexity curve of the FRSB spherical model which is the only FRSB model known 
to be non-chaotic in temperature [51] at variance with the SK model |25| . 

Starting from the function S(/, y) we could obtain the total linear-susceptibility using the level crossing 
argument. Indeed to obtain a description of the evolution of the states at first order in h y it was sufficient 
to consider £(/, y). To obtain the next order we must consider the complexity £(/, y, Xy), where \y 1S the 
intrastate susceptibility associated to the field h y . Then the associated zero-complexity line / = f(y,Xy) 
must be used in eq. (|T7|) . Extremising with respect to y and \y we can obtain the value of third derivative 
of the TAP free energy with respect to the external field h y . Higher orders are obtained in the same way, 

in general to obtain the /c-th derivative of the TAP free energy we need £(/, y, Xyi ■ ■ ■ ,Xy k )j *- e - the 
complexity as a function of the intrastate susceptibilities up to order k — 1. 

In the present paper we focused on the evolution of density of TAP states under variation of exter- 
nal parameters over a small but finite range Ah. As discussed in section 2, if we go down to scale of 
8h ~ 0(1/ \/N), we will observe individual level-crossings whose characters are strongly non self-averaging. 
Presumably this is relevant for problems of heterogenous thermal fluctuations and responses at mesoscopic 
scales [SUJEHI^j some of which have now become accessible experimentally. Further investigation of the 
intermediate scales between 8h and Ah will be interesting in this respect 11 . 

Acknowledgments: It is a pleasure to thank G. Parisi for many stimulating discussions and especially 
for pointing us the argument reported at the end of section 2. 



Appendix A 

In this appendix we show how to compute the function $(A) which is basically the Legendre transform of 
the zero-complexity curve / = by 2 . In the following we assume that there exist a local function y(rrii, qEA) 
such that the parameter y can be expressed as y a = J^. y(mf , q aa )/N. This includes the case of the 
magnetization and of the entropy. The computation of the function &(\ y ) can be done following standard 
techniques for computing averages over TAP solutions, we present the result in the case of the SK model 
and skip the details of the derivation, which are largely described in the literature (see e.g. Ref. |12l I23| ). 
In order to further simplify the presentation we report the expression of ^(A, 0) defined as: 

d>(A„0) = ilnp ; p = £ e V^ (36) 

a 

the quenched disorder average of <S>(X y , 0) can be computed through the replica method: 



$(A„,0) = lim -lnp" (37) 

Using the supersymmetric formulation of Ref. |12| the disorder average of p n can be expressed as an integral 
over eight macroscopic bosonic and fermionic variables = {r a b^t a b,qab,^ab,7>ab^Pab,'PabiPab} 

W = J dQexp[N^ l) +7Vx4 n) ] (38) 

Where the action is specified by: 



^ = -Kbqab - + + VabVab + ZpabPab + PabPab ( ) 
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and 



log 



Y\_dm a dx a d^ a dip a exp x a (j>x{q aa , m a ) + ip a ip a <l>2(q aa ) T^a) 



q a bf3 2 X a X b 



r a bm a x b + t ab ip a ijj b + X 



- [iabl3m a ip b - ^ a m b p ab (3 - p ab (3x a ip b - ^ a x b \x ab fi + X y y{m a ,q aa ) 

and the functions (j)i(q,m) and </> 2 (g, to) are given by: 

01 (q, m) 
<t>2(q,m) 



(3 2 (l - q)m + tanh 1 (m) 
P 2 (l~q)+ ' 



1 — TO 



(40) 

(41) 
(42) 



Note that the only modification with respect to the standard computation (i.e. X y = 0) is in the presence 
of the term X y y(m ai q a a) in the integral in . The second derivative of ^(Aj,, 0) at Aj, = is given by: 



<9 2 $ 
l 9Af 



2v(") 



d 2 E 



N 



<9£ 



9E 



(«)' 



(43) 



Where the square brackets mean average with respect to the action eq. (|38|l and: 



<9£ 



(») 



8Xy 



9 2 E 



9A 2 



((^J7(77l Q , gaa ))) 
a 

((^2y(m a ,q aa )y(m b ,q bb ))) - ((^y(m a ,q aa ))f 



ab 



(44) 
(45) 

(n) 

Where the double brackets mean average performed with respect to the integrand in the definition of . 
The previous averages must be evaluated at X y = 0. The action Oilcan be evaluated through a saddle-point 
method. Note that in general to evaluate the the second term in eq. (|43|l we need to study the Hessian of 
the saddle-point which in general is very complicated. However in the case of magnetic field perturbation 
in zero field we have y(m a , q aa ) = m a and dT,^ / dX m at X rn = is identically zero for symmetry reasons, 
therefore only the first term survives and we don't need to compute the Hessian of the SP. Thus only the 
first term in the r.h.s. of eq. (|45|l contributes to the second derivative of &(X y , 0) and we recover the result 



<9 2 $ 



dXl 



lim -( ((y"m Q 777 fc )) ) = q E A 



(46) 



A m =0 ab 

Where we have used that SP equations q a b = ((J2ab mamb )) 



Appendix B 

In this appendix we show how to compute the sample-to-sample fluctuation of the TAP susceptibility eq. 
Q following the similar computation for the true thermodynamic susceptibility. The first term is the 
intrastate susceptibility and does not fluctuate with the disorder, analytically this is a consequence of the 
fact that it is a single replica quantity ^ |S] . The second term is the fluctuation of the total magnetization 
over all TAP solutions A -1 [J2i 3 ■(mi'mj) — ( m i)( m j)}, in order to check if it is self-averaging we compute 
the average of its square. The computation can be done along the lines of the same replica computation 
of the thermodynamic susceptibility fluctuations 5 . The objects one needs to compute are averages of 
the form {rrii t irnj t i)TAp('rni t 2}TAp('rnj J 3)TAP where 1, 2, 3 are different replicas with the same realization of 
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the disorder where the square brackets mean summation over all TAP states with the Boltzmann weight. 
Introducing source fields in the definition of p = ^2 a exp[—(3f a + Ximi ta ] this can be written as: 



Now we multiply the quantity in the above disorder average by a factor p n and divide the whole average 
by p"; taking the limit n — > the result does not change, therefore we can write: 



<:) 



4 



(mi A m jA )TAp(mi,2)TAp(mj, 3 )TAP = hm — — — — — — — — hip™ (48) 

n— >0 OAi iUAj t iUAi_20Ai_ 3 

the expression of p n in presence of the source field can be computed as in appendix A, the result is: 

W= J deexp[JVE£ n) +ArE| n) ]{(e A< ' imi ' 1+Ai ' 3m *- 3 ))((e A ^ im ^ 1+ ^ 2m '- 2 )) (49) 

the derivative is: 



(m^imj^TApimi^TApimj^TAP = lim( ((mi,imi,3»({mj,imj,2)) ) (50) 

ra— >0 

Where the meaning of the double square brackets and of the square bracket is the same in appendix A. In 
the thermodynamic limit this quantities can be averaged by the saddle point method, in particular using 
the saddle point equation with respect to A b we get: 



m, 



Amj.^TApfai^TApimj^TAP = 913912 (51) 



This must be summed over the different SP, instead we can evaluate on a single SP the same object under 
all possible permutations of the replica indices: 



1 



(m^imj^TApimi^TApimj^TAP = —, 777 ;rc V" q a bq ac (52) 

n (n — 1 ri — I) L — ' 

KM 

All the various terms can be evaluated with this method and at the end it turns out that the r.h.s. of eq. 
Q is not self-averaging. Furthermore, as shown in jS], at the lower band edge the matrix q a b of the TAP 
computation coincide with the Parisi solution and one can show that its disorder variance is equal to that 
of the thermodynamic susceptibility computed in [5J- 
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